Information processing device, information processing method and information processing system

ABSTRACT

An information processing device according to the present technology includes a determination unit that determines importance related to a cell-specific event of a cell, using image data obtained from a time-series imaging process targeting the cell The information processing device also includes a control unit that controls a process regarding a setting for a target of acquisition of image data in the time-series imaging, on the basis of a determination result of the importance.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2017/040326 filed on Nov. 8, 2017, which claimspriority benefit of Japanese Patent Application No. JP 2016-239254 filedin the Japan Patent Office on Dec. 9, 2016. Each of the above-referencedapplications is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing device, aninformation processing method and an information processing system.

BACKGROUND ART

In the fields of medical care and life science, a variety of cells aresubjected to image capturing, and motion or change of state of the cellsare observed on the basis of the acquired image data. Accumulation ofsuch image data may make it difficult to search image data thatrepresent cell-specific events (for example, cell stage, cleavageoccurring at boundaries between cell stages, morphological changes suchas cell division, or degeneration) regarding a target cell. In somecases, image data would no more be stored due to strain on the capacityof storage for accumulating image data, possibly missing a chance toacquire image data that represents a valuable event regarding the cell.

Hence, technologies capable of more exactly acquiring image data assumedas a target of acquisition are under development. For example, PatentLiterature 1 below discloses a technique by which importance of amedical image is determined under predetermined conditions, in responseto an operation signal generated by user's operation, and the medicalimage is then compressed according to a compressibility of the medicalimage determined on the basis of such importance. On the other hand,Patent Literature 2 below discloses a technique by which stored videodata is erased on the basis of loudness or priority (importance) set byuser's operation.

CITATION LIST Patent Literature

[Patent Literature 1] JP 2014-42727A [Patent Literature 2] JP2006-254270A

DISCLOSURE OF INVENTION Technical Problem

Observation of motion or change of state of the cells, or analysis andevaluation of cell-related events may sometimes take a long period suchas several days. This results in production of a huge volume of imagedata during the observation, overloading the user who manages todetermine importance of the image data. It is therefore difficult forthe techniques disclosed in aforementioned Patent Literature 1 andPatent Literature 2 to exactly and efficiently store the image data ofcells which are acquired by image capturing over a long period.

Hence the present disclosure is to propose a novel and improvedinformation processing device, an information processing method and aninformation processing system capable of exactly and efficiently storeimage data of cells, without overloading the user.

Solution to Problem

According to the present disclosure, there is provided an informationprocessing device including: a determination unit that determinesimportance related to a cell-specific event of a cell, using image dataobtained from a time-series imaging process targeting the cell; and acontrol unit that controls a process regarding setting for a target ofacquisition of image data in the time-series imaging process, on thebasis of a determination result of the importance.

Moreover, according to the present disclosure, there is provided aninformation processing method including by a processor: determiningimportance related to a cell-specific event of a cell, using image dataobtained from a time-series imaging process targeting the cell; andcontrolling a process regarding setting for a target of acquisition ofimage data in the time-series imaging process, on the basis of adetermination result of the importance.

Moreover, according to the present disclosure, there is provided aninformation processing system including: an imaging device that includesan imaging unit that produces an image by image capturing; and aninformation processing device that includes a determination unit thatdetermines importance related to a cell-specific event of a cell, usingimage data obtained from a time-series imaging process targeting thecell by the imaging unit; and a control unit that controls a processregarding setting for a target of acquisition of image data in thetime-series imaging process, on the basis of a determination result ofthe importance.

Advantageous Effects of Invention

As explained above, the present disclosure makes it possible to storeimage data of cells in a certain and efficient manner, withoutoverloading the user.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an overview of a configuration of aninformation processing system according to an embodiment of the presentdisclosure.

FIG. 2 is a functional block diagram illustrating an exemplaryfunctional configuration of an information processing device accordingto a first embodiment of the present disclosure.

FIG. 3 is a drawing explaining an exemplary setting of a region ofinterest in an image analysis unit according to this embodiment.

FIG. 4 is a schematic drawing illustrating an exemplary flow of embryodevelopment stages.

FIG. 5 is a graph illustrating an exemplary relation between embryodevelopment stages and importance in a timeline of imaging process.

FIG. 6 is a graph illustrating an exemplary updating process for thetimeline at determination time point t₁ carried out by the setting unitaccording to this embodiment.

FIG. 7 is a graph illustrating an exemplary updating process for thetimeline at determination time point t₂ carried out by the setting unitaccording to this embodiment.

FIG. 8 is a graph illustrating an exemplary updating process for thetimeline at determination time point t₃ carried out by the setting unitaccording to this embodiment.

FIG. 9 is a graph illustrating a first example of the setting for atarget of generation, carried out by the setting unit according to thisembodiment.

FIG. 10 is a graph illustrating a second example of the setting for atarget of generation, carried out by the setting unit according to thisembodiment.

FIG. 11 is a graph illustrating a third example of the setting for atarget of generation, carried out by the setting unit according to thisembodiment.

FIG. 12 is a flowchart illustrating an exemplary process performed by aninformation processing device according to the embodiment.

FIG. 13 is a graph illustrating an exemplary setting for a target ofgeneration, carried out by the setting unit according to the secondembodiment of the present disclosure.

FIG. 14 is a flowchart illustrating an exemplary process performed by aninformation processing device according to the embodiment.

FIG. 15 is a block diagram illustrating an exemplary hardwareconfiguration of an information processing device according to anembodiment of the present disclosure.

MODE (S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment (s) of the present disclosure willbe described in detail with reference to the appended drawings. Notethat, in this specification and the appended drawings, structuralelements that have substantially the same function and structure aredenoted with the same reference numerals, and repeated explanation ofthese structural elements is omitted.

Now, the description will be made following the order below.

1. Overview of Information Processing System

2. First Embodiment

2.1. Exemplary Configuration

2.2. Exemplary Processing

3. Second Embodiment

3.1. Exemplary Configuration

3.2. Exemplary Processing

4. Exemplary Hardware Configuration

5. Conclusion

FIG. 1 is a diagram illustrating an overview of a configuration of aninformation processing system 1 according to an embodiment of thepresent disclosure. As illustrated in FIG. 1, the information processingsystem 1 has an imaging device 10, and an information processing device20. The imaging device 10 and the information processing device 20 areconnected through a variety of wired or wireless networks.

(Imaging Device)

The imaging device 10 is a device that produces images (for example,still images or video) by image capturing. The imaging device 10according to the embodiment is typically embodied by a digital camera.The imaging device 10 may alternatively be embodied by any of deviceswith imaging function, such as smartphone, tablet, game machine orwearable device.

The imaging device 10 according to the embodiment is provided, asillustrated in FIG. 1, inside a culture incubator I1, above a dish D1that contains a medium M1 in which cell to be observed is cultured. Theimaging device 10 captures images of the cell cultured in the medium M1at a predetermined frame rate, to thereby produce image data. Now, in acase where a plurality of cells is present in the dish D1, the cells maybe captured so as to be contained one by one in imaging frames, or maybe captured so that a plurality of cells is contained in a singleimaging frame. On the other hand, in a case where a plurality of cellsis individually cultured in a plurality of dishes, it is also possibleto capture images of each cell, while suitably moving the imaging device10 or the dishes using a freely selectable driving unit provided to theincubator I1.

Now, the imaging device 10 may be provided inside the incubator I1, oroutside the incubator I1. Alternatively, the imaging device 10 isapplicable to image capturing of cells which are not housed in theincubator I1. Still alternatively, the imaging device 10 may be providedintegrally with the incubator I1.

In addition, the incubator I1 is not specifically limited in terms ofspecification or size, thus allowing use of any incubator which iscapable of providing an environment suitable for culturing cell. Alsoregarding the dish D1 and the medium M1, those publicly-known to besuitable for culturing cell are employable.

In more details, the imaging device 10 according to the embodiment hasan imaging unit 101 and an imaging control unit 102, as illustrated inFIG. 1.

The imaging unit 101 has various components including an image sensorsuch as CCD (Charge Coupled Device) or CMOS (Complementary Metal OxideSemiconductor), a lens that controls focusing of an object image on theimage sensor, and a light source that illuminates an object, andcaptures images of real space using these components.

In order to correctly specify motion inside the cell to be observed, theimaging unit 101 according to the embodiment captures images of apredetermined imaging region that contains cell (s) cultured on themedium M1. The imaging unit 101 may capture images of cell directly(without being interposed by other component, such as lens), or maycapture images of cell while being interposed by other component such asa microscope with an objective lens. In this design, the objective lenspreferably has a magnification of 40× to 60× or around, for the purposeof capturing motion of cell in sub-micron order. While the frame rate isnot specifically limited, it is preferably set depending on the extentof changes of a target to be observed. More specifically, the frame rateis preferably set to a value so that motion of cell in sub-second ordermay be captured.

The imaging unit 101 may be provided with a plurality of imagingmodules. More specifically, the imaging device 10 may be provided withan imaging module for time-lapse capturing and an imaging module forvideo shooting described later. With such design, it now becomespossible to produce image data that satisfy performances, includingpicture quality, individually required for the time-lapse images andvideo.

A signal generated as a result of imaging process by the imaging unit101 is output to the imaging control unit 102.

The imaging control unit 102 has a processing circuit built up with aCPU (Central Processing Unit), ROM (Read Only Memory), a RAM (RandomAccess Memory) and so forth; and a communication device, and controlsentire operations of the imaging unit 101. The imaging control unit 102typically controls capturing by the imaging unit 101, and generatesimage data on the basis of a signal obtained from the capturing process.

For example, the imaging control unit 102 can control the timing theimaging unit 101 carries out the imaging process. More specifically, theimaging control unit 102 can control the imaging unit 101 to carry outthe imaging process intermittently at predetermined intervals(so-called, time-lapse imaging). The imaging control unit 102 can alsocontrol the imaging unit 101 to carry out imaging process continuouslyover a predetermined period so as to produce video data. Alternatively,the imaging control unit 102 may change, add, or delete the timingaccording to which the imaging unit 101 carries out image capturing ofcells, in response to a command output from the information processingdevice 20, which will be detailed later.

Meanwhile, in a case where a plurality of cells is captured, the imagingcontrol unit 102 may directly or indirectly control the incubator I1 soas to move the imaging device 10 or the dish according to the capturetiming of the cells to be captured. Note that an exemplary control ofthe timing of capturing process according to one embodiment of thepresent disclosure will be described later.

Alternatively, the imaging control unit 102 may control the wavelength,illumination intensity or illumination time of the light source providedto the imaging unit 101. For example, the imaging control unit 102 maycontrol the light source of the imaging unit 101, so as to illuminatecell with light of appropriate wavelength at minimum illuminationintensity, only within a period the imaging unit 101 is capturingimages. This can minimize phototoxicity on cell.

Still alternatively, the imaging control unit 102 may allow the imagingunit 101 to carry out the imaging process, while controlling thedistance between the dish D1 and the imaging device 10, typically withthe aid of a stage provided to the incubator I1. In this way, sliceimages (so-called, Z-stack image) or confocal image assumed as thetarget to be observed may be produced. This enables three dimensionalstructural analysis of the target to be observed.

Now as will be detailed later, the imaging control unit 102 maypreliminarily set a region of interest (ROI) on images. The region ofinterest in this context means a region subjected to the later-describedimage analysis carried out by the image analysis unit 202. The region ofinterest in this embodiment is a region corresponded, for example, tointracellular tissues (cytoplasm, nucleus, etc.) or peripheral tissues(cell membrane). More specifically, if the target to be observed is anembryo, the region of interest is preferably an internal region of theembryo, and particularly a region corresponded to cytoplasm of theembryo. How to set the region of interest and so forth will be describedlater.

The imaging control unit 102 outputs the produced image data and soforth to the information processing device 20. Note such image data maybe recorded in an unillustrated storage owned by the imaging device 10,or may be recorded in an external server, cloud or storage differentfrom the information processing device 20.

(Information Processing Device)

The information processing device 20 is a device having an imageanalyzing function. The information processing device 20 may be embodiedby various devices having image analyzing function, including PC(Personal Computer), tablet and smartphone. The information processingdevice 20 contains a processing circuit such as CPU (Central ProcessingUnit), and a communication device which includes hardware allowed forwireless or wired communication. For example, in the informationprocessing device 20 according to this embodiment, the communicationdevice acquires, from the imaging device 10, image data targeted at acell obtained in the time-series imaging process. The processing circuitthen determines, on the basis of the acquired image data, importancerelated to a cell-specific event regarding the cell. The processingcircuit then controls, on the basis of the determination result ofimportance, a process regarding setting for a target of acquisition ofimage data in the time-series imaging process. Result of such process isoutput typically to the imaging device 10, or to the internal orexternal storage device of the information processing device 20. Notethat the information processing device 20 may be embodied by one or aplurality of information processing devices on a network. A functionalconfiguration for realizing the respective functions of the informationprocessing device 20 will be described below.

Note that, although the information processing system 1 in theembodiment includes the imaging device 10 and the information processingdevice 20, the present technology is not limited to this design. Forexample, the imaging device 10 may take part in processing regarding theinformation processing device 20 (for example, determination process,setting process, or the like). In this design, the informationprocessing system 1 is embodies by an imaging device having, forexample, a determination function.

Now the cell, which is an exemplary cell to be applied by theinformation processing system 1 according to the embodiment, will beexplained. For example, a normal human fertilized egg shows a pronucleusthat appears immediately after fertilization, and then starts celldivision. Note that, although the fertilized egg is not an embryo butpronuclear-stage embryo in the strict sense, the present specificationwill also deal the fertilized egg as one form of embryo.

In cleavage, a normal fertilized egg (embryo) initially in the 1-cellstage cleaves to produce a 2-cell stage embryo, then repeats cleavage toproduce a 4-cell embryo, 8-cell embryo, morula, to reach blastocystfinally over several days. In the normal course, the blastocyst adheresto a uterus, the pellucida that surrounds the embryo breaks, and theembryo thus hatches. That is, the timing of cleavage can be deemed to beone checkpoint of embryogenetic stage. The cleavage and the eventsdemonstrated in embryo development stages are examples of thecell-specific event. As is clear from above, the cell-specific eventmeans any events that are demonstrated by cells on the basis ofphysiological function intrinsically or genetically owned by the cells.The cell-specific event may include events demonstrated in response tophysical, chemical or electromagnetic stimulation. For example, also anevent demonstrated as a result of medicinal effect of a drugadministered to the cell may be included in the cell-specific event.

As a technique for evaluating growth of embryo, having been developedis, for example, a technique of observing or analyzing image dataobtained by successively capturing images of embryo. Evaluation ofgrowth of embryo, however, takes at least several days, during which ahuge volume of image data generated in the imaging process will beaccumulated in the storage or the like. This makes it difficult tosearch among the huge volume of image data to find out image data thatrepresents a target event regarding the embryo. Moreover, strain on thecapacity of storage or the like may possibly inhibit storage of imagedata. In particular in observation of cells such as embryo, overwritingof past image data is not desirable, from the viewpoint of supportingaccuracy of the evaluation.

As a countermeasure for the aforementioned problem, for example, JP2014-42727A and JP 2006-254270A disclose techniques of compressing ordeleting image data, on the basis of importance (priority) preset byuser's operation or the like. Culture duration of cells such as embryo,however, ranges at least over several days as described above. Imagedata produced in the duration will be huge, and will overload the userwith a task to determine which image data need be compressed or deleted(that is, how to preset importance).

Now, the information processing system 1 according to this embodimentdetermines importance related to a cell-specific event regarding a cell,using image data generated in the time-series imaging process targetingthe cell, and controls a process regarding the setting for a target ofacquisition of image data in the time-series imaging process, on thebasis of the determination result of importance. As will be explained inthe embodiments below, the process regarding the setting for a target ofacquisition of image data includes a process of presetting time point ofgeneration of image data (that is, preset of timing of image capturingof cells in a real-time processing), and determination of a target ofstorage of the produced image data (that is, determination of image datato be stored or deleted in post-processing). In this way, the importancemay be determined on the basis of the event regarding cell contained inthe image data, making it possible to efficiently and exactly storeimage data regarding event specially desired to be observed by the user.Hence, it now becomes possible to evaluate motion or change of state ofcells even after the imaging process ranged over a long period, withoutoverloading the user.

The overview of the information processing system 1 according to oneembodiment of the present disclosure has been described above. Now theindividual embodiments below will explain exemplary applications of thepresent technology, focusing embryo as one example of the cell. Thepresent technology is, however, not limited to these examples. Forexample, the information processing system 1 according to the embodimentis also applicable to cell, biotissue and so forth capable ofdemonstrating morphological changes specific to living bodies, such asthose causing cell division, or such as those incorporating other cellor the like. In addition, besides embryonic cleavage, the eventsregarding cells to which the present technique is applied can includeproliferation or division of cancer cell, nerve cell, ES cell, iPS celland other cells; differentiation of stem cell; and morphological changesof immune cell and other cells. In addition, the target to which theinformation processing system 1 according to this embodiment is appliedmay be not only cells, but may also be biological samples such asbiological tissues. Alternatively, the target to which the informationprocessing system 1 according to this embodiment is applied may beanimal, plant or inanimate structures. For example, any structural ormorphological changes that will occur over several hours or several dayson the basis of events specific to the target to be observed, includingproliferation or metamorphosis of living body, or growth of thin film ornano-cluster crystal, are possible targets of application of theinformation processing system 1 according to this embodiment.

The information processing device 20 contained in the informationprocessing system 1 according to one embodiment of the presentdisclosure is embodied in the embodiment below. Specific examples ofconfiguration and processing of the information processing device 20will be explained below.

2. FIRST EMBODIMENT

The first embodiment of the present disclosure will be explainedreferring to FIG. 2 to FIG. 12. In the information processing system 1according to this embodiment, there are a determination process ofimportance using the image data obtained from the imaging process, and aprocess regarding the setting for a target of acquisition of image datain the imaging process on the basis of importance, which are carried outconcurrently with the imaging process in the imaging device 10. In thisway, it now becomes possible to selectively produce image dataconsidered to be important in the aforementioned imaging process. Theimage data may therefore be stored efficiently and more exactly, withoutoverloading the user.

<2.1. Exemplary Configuration>

FIG. 2 is a functional block diagram illustrating an exemplaryfunctional configuration of the information processing device 20according to the first embodiment of the present disclosure. Asillustrated in FIG. 2, the information processing device 20 according tothe embodiment has a control unit 200, a communication unit 210 and astorage unit 220. The individual function units will be explained below.

(Control Unit)

The control unit 200 controls entire operations of the informationprocessing device 20. Further, as illustrated in FIG. 2, the controlunit 200 has the individual functions of an image acquisition unit 201,an image analysis unit 202, a determination unit 203, a setting unit204, and an output control unit 205, and controls operations of theinformation processing device 20 according to the embodiment in aleading manner. The functions of the individual functional unitscontained in the control unit 200 will be described later. The controlunit 200 is typically embodied by a processing circuit such as CPU.

(Communication Unit)

The communication unit 210 is a communication means possessed by theinformation processing device 20, and takes part in various types ofcommunications in a wired or wireless manner, through a network (ordirectly), with an external device. For example, the communication unit210 communicates with the imaging device 10. More specifically, thecommunication unit 210 acquires image data produced by the imagingdevice 10. Alternatively, the communication unit 210 may communicatewith a device other than the imaging device 10. For example, thecommunication unit 210 may send images acquired by the image acquisitionunit 201, information regarding proposal controlled by the outputcontrol unit 205, and so forth to an external display device or thelike. The communication unit 210 is typically embodied by acommunication device such as communication antenna combined with RF(Radio Frequency) circuit (wireless communication), an IEEE 802.15.1port combined with a transceiver circuit (wireless communication), anIEEE 802.11b port combined with a transceiver circuit (wirelesscommunication), or a LAN (Local Area Network) terminal combined with atransceiver circuit (wired communication).

(Storage Unit)

The storage unit 220 is a storage device installed in the informationprocessing device 20 and stores information acquired by thecommunication unit 210, information obtained by the respective functionunits of the control unit 200, and the like. For example, the storageunit 220 can store the image data acquired from the imaging device 10through the communication unit 210. Further, the storage unit 220appropriately outputs the stored information in response to a requestfrom each function unit of the control unit 200 or from thecommunication unit 210. In addition, the storage unit 220 carries out aprocess regarding image data stored in the storage unit 220 (forexample, output of image data to the control unit 200, acquisition ofimage data from the control unit 200, and deletion of image data), inresponse to a command issued from the control unit 200. The storage unit220 is typically embodied by a magnetic recording medium such as harddisk, or a nonvolatile memory such as flash memory. Alternatively, thestorage unit 220 may be embodied for example by an external cloud serveror storage. In this design, the information processing device 20 is notnecessarily provided with the storage unit 220.

Next, the functions of the respective function units installed in thecontrol unit 200 will be described.

(Image Acquisition Unit)

The image acquisition unit 201 has a function that acts to acquire, fromthe imaging device 10, image data produced in the time-series imagingprocess targeting the cell. The image data is acquired through thecommunication unit 210.

The image data produced in the time-series imaging process in thiscontext means, for example, image data of time-lapse image. Thetime-lapse image is a series of still images obtained from the imagingprocess carried out intermittently at predetermined intervals.

Examples of preset conditions for the imaging process for the time-lapseimage are listed in Table 1. In blastocyst analysis of embryo (analysisof embryonic growth) assumed as a target to be observed according tothis embodiment, the time-lapse image is obtained by repetitivelycapturing image of embryo at 15 minute intervals. The imaging processcan be continued, for example, over 5 days. Conditions for such imagingprocess of time-lapse image can suitably be preset depending on detailsof analysis of the target to be observed as listed in Table 1. Forexample, in the imaging process for analyzing migration ability ofcancer cell, the imaging intervals can be one hour, and the processperiod can be one week. Meanwhile, in the imaging process for analyzingcocultivation with iPS cell, the imaging intervals can be 10 minutes,and the process period can be one month.

TABLE 1 Analysis of Analysis of Analysis of target to Analysis ofmigration ability cocultivation be observed blastocyst of cancer cellwith iPS cell Imaging intervals 15 Minutes 1 Hour 10 Minutes Processperiod of 5 Days 1 Week 1 Month imaging process Continuous imaging 10Seconds 10 Seconds 10 Seconds time (of video)

On the basis of such conditions for the imaging process, a timelineregarding the imaging process of time-lapse image is set. The timelineaccording to this embodiment means time-series information thatdetermines a target of generation of image data (target of acquisition)in the time-series imaging process, which is preset from the viewpointof data size. The timeline can contain timing of imaging in the imagingprocess, number of shooting (Z-stack image) per a single timing ofimaging, or information regarding resolution or compressibility of theimage data that is expected to be produced.

For example, the timeline regarding analysis of embryo developmentstages listed in Table 1 is a timeline that represents image capturingrepeated at 15 minute intervals over a process period of imaging processof 5 days. Such timeline may suitably be modified by the setting unit204 described later.

Note that the aforementioned image data is not limited to the time-lapseimage, but may be image data of video. The video contains images in aplurality of successive frames obtained by continuous image shootingover a predetermined period. Such video may be a video obtained byseamless image shooting that takes place from the start through the endof shooting, but may preferably be a video obtained by continuous imageshooting that takes place only for a predetermined period, and atpredetermined intervals, taking phototoxicity and image processing loadinto consideration. In this embodiment, the plurality of images thatcomposes the video can be those produced by continuous image shootingthat takes place over several seconds to several tens of seconds, at aframe rate of several to several tens of frames per second.

For example, as listed in Table 1, in the analysis of embryo developmentstages assumed as the target to be observed according to thisembodiment, video is obtained by 10-second-long continuous shooting ofembryo, repeated at 15 minute intervals. In this case, theaforementioned time-lapse image may be an image of one frame extractedfrom a plurality of imaging frames that compose the video. The thusextracted image of one frame may be, for example, an image in the firstframe, among from a plurality of imaging frames that composes the video.

Note that the image data may be image data regarding one or a pluralityof embryos. The image data regarding the plurality of embryos mean imagedata that contain the plurality of embryos one by one in imaging frames,or image data that contains the plurality of embryos in a single imagingframe.

The image acquisition unit 201 according to this embodiment acquires,for example, image data that contains an embryo captured by the imagingunit 101 of the imaging device 10. More specifically, the imageacquisition unit 201 according to this embodiment can acquire, throughthe communication unit 210, an image that contains an embryo captured ata predetermined timing of imaging in a real-time manner by the imagingunit 101 of the imaging device 10, according to a preset timeline. Inthis case, the determination process for determining importance, and thesetting process for setting the target of generation of image data inthe individual functional units in the succeeding stage may be carriedout in a real-time manner.

Note that although the timeline according to this embodiment will beexplained assuming that it will be used for controlling the timing ofimaging in the imaging process carried out by the imaging control unit102 of the imaging device 10, the present technology is not limited tothis example. That is, the timeline may alternatively be used fordetermining image data to be acquired from the imaging device 10 of theimage acquisition unit 201. Note, however, that for the case of settingthe timeline for increasing the timing of imaging as described later,the preset timeline can be used to control the timing of imaging in theimaging process carried out by the imaging control unit 102.

Note that, for the purpose improving accuracy of the determinationprocess, the image acquisition unit 201 may properly carry outcalculation typically for interpolation, noise removal, and correctionsuch as rotation, of the acquired image.

The image acquisition unit 201 outputs the acquired image data to theimage analysis unit 202.

(Image Analysis Unit)

The image analysis unit 202 has a function of analyzing the image data.For example, the image analysis unit 202 can set a region of interest(ROI) on an image of embryo contained in the image data, can carry outimage analysis in the region of interest, and can calculate feature ofthe region of interest.

The region of interest means a region to be analyzed in the succeedingprocess, in an occupied area of the image. FIG. 3 is a drawing forexplaining an exemplary setting of a region of interest in the imageanalysis unit 202 according to this embodiment. As illustrated in FIG.3, the region of interest R1 according to this embodiment is a regionthat corresponds to the inside of embryo E1 contained in the image F1.The inside of embryo E1 may specifically mean the cytoplasm contained ina central part of embryo E1. With such design, it now becomes possibleto specify motion of the embryonic cytoplasm by an analytical process orthe like in the succeeding stage. Note that, for an exemplary case wherealso geometrical changes in embryo need be analyzed, the region ofinterest may be defined not only by cytoplasm, but also by pellucida (apart assumed as an interface to the outside field).

The image analysis unit 202 may preset the region of interest, typicallyaccording to operation by the user made on an unillustrated input device(for example, known input devices such as mouse, touch pen and touchpanel). Alternatively, the image analysis unit 202 may preset the regionof interest, by using freely selectable image analysis technologiestypically based on known algorithms including image thresholding, Houghtransformation and machine learning. Still alternatively, the imageanalysis unit 202 may preset the region of interest for a plurality ofimages, by estimating how the region of interest preset to a singleimage can move over the plurality of images, using an algorithm such asoptical flow. With such design, it now becomes possible to automaticallypreset the region of interest for the plurality of images in which themotion inside the embryo will be analyzed.

Alternatively, processes for presetting the region of interest for videomay, for example, be performed preliminarily by other device havinginformation processing function, such as the imaging device 10. In thiscase, the function of the image analysis unit 202, regarding presettingof the region of interest, may be left unused.

The image analysis unit 202 can carry out image analysis in the presetregion of interest regarding embryo, and can calculate feature of theregion of interest. The feature includes, for example, feature based onmorphology of the region of interest, feature based on motion (kinetics)of the region of interest, and feature based on pixel information ofimage. The morphology of the region of interest means morphology ofembryo that corresponds to the region of interest, or mode of the insideof embryo (cytoplasm), and the feature based on the morphology of theregion of interest is a feature obtained from one still image. On theother hand, the motion of the region of interest includes motionresulted from morphological change of embryo that corresponds to theregion of interest, and from modal change inside embryo that correspondsto the region of interest, and the feature is obtained from a pluralityof images.

The feature based on morphology of the region of interest is exemplifiedby area, circumferential length, circularity, long axial length or shortaxial length of the region of interest; or, changes in average, dynamicrange, variance or standard deviation of luminance. Meanwhile, thefeature based on motion of the region of interest is exemplified byfeature that represents morphological change of embryo that correspondsto the region of interest, and feature that represents modal changeinside embryo. More specifically, the feature that representsmorphological change of embryo that corresponds to the region ofinterest is exemplified by area, circumferential length, circularity,long axial length or short axial length of the region of interest; or,changes in average, dynamic range, variance or standard deviation ofluminance. Meanwhile, the feature that represents modal change insideembryo that corresponds to the region of interest is exemplified byaverage, acceleration, standard deviation, travel range, maximum value,minimum value or median of motion within the region of interest.Furthermore, the feature based on pixel information of image isexemplified by luminance histogram or frequency spectrum.

In this embodiment, a plurality of features may be calculated by imageanalysis. For example, the image analysis unit 202 may calculate thefeature based on morphology of the region of interest (for example,variance of image), feature that represents modal change inside embryothat corresponds to the region of interest (for example, average ofmotion), and feature that represents morphological change of embryo thatcorresponds to the region of interest (for example, change in thecircumferential length of the region of interest). Acquisition of aplurality of features will improve accuracy of determination in thedetermination unit 203 in the succeeding stage.

Such image analysis can be performed using any of known techniques. Forexample, motion vector may be analyzed in order to calculate the motionsize within the region of interest. The motion vector can be acquiredusing any of known algorithms such as block matching method or gradientmethod. Alternatively, morphology and so forth in the region of interestmay be analyzed by any of known techniques, on the basis of pixelinformation of the image.

The image analysis unit 202 outputs the featured obtained by the imageanalysis to the determination unit 203.

(Determination Unit)

The determination unit 203 has a function of determining importancerelated to the cell-specific event regarding cell, using the image data.

The determination unit 203 according to this embodiment can discriminatethe embryo-specific event regarding embryo, on the basis of the featurecalculated by the image analysis unit 202. Then the determination unit203 according to this embodiment can determine importance on the basisof such determination result.

Now the embryo-specific event regarding embryo and the importance willbe explained. The embryo-specific event regarding embryo according tothis embodiment means embryo development stages. FIG. 4 is a schematicdrawing illustrating an exemplary flow of embryo development stages. InFIG. 4, states A1 to A6 respectively represent embryo development stageslisted in Table 2 below. Note that there may be a stage of morulabetween states A5, A4 b and A6. Also note that in a case where statesimmediately before and immediately after cleavage are discriminable evenwithin the same cell stage, the embryo development stage can beclassified, for example, into early period or late period of the cellstage. Such ranks of stages can appropriately be preset depending on thecell-specific event.

TABLE 2 State Embryo development stages A1 1-Cell stage A2 2-Cell stageA3 4-Cell stage A4a Cleavage in process A5 8-Cell stage A4b Cleavage inprocess A6 Blastocyst

The importance determined in this embodiment means importance related tothe embryo development stages determined from image data. An assumablecase is that the 2-cell stage (state A2) and the 4-cell stage (stateA3), enclosed by a broken line in FIG. 4, are desired to be observed ingreater detail. In this case, if an image of embryo contained in imagedata is determined to be in state A2 or state A3, the determination unit203 assigns high importance to the image data. Note that the number ofranks or weight that represents the degree of importance mayappropriately be set. This embodiment will be explained referring tocases where the importance is ranked on a three-point scale of “high”,“medium” and “low”, or on a two-point scale of “high” and “low”. Therank of importance is, however, not limited to this example.

FIG. 5 is a graph illustrating an exemplary relation between the embryodevelopment stages and the importance in the timeline of the imagingprocess. Notations of A1 to A6 in FIG. 5 correspond to the states thatrepresent the embryo development stages shown in FIG. 4 and Table 2. Forexample, as shown in Timeline 1, a possible way is that high importanceis assigned to time zone 1000 b that represents the embryo developmentstages given by state A2 and state A3, and (relatively) low importanceis assigned to the residual time zones 1000 a and 1000 c. In this way,the setting unit 204 and the output control unit 205 in the succeedingstage are now enabled to preferentially produce image data and so forthin the time zone assigned with high importance.

Alternatively, for the case as illustrated in Timeline 2 where the imagedata regarding the embryo development stages in state A4 a andthereafter are not necessary, a possible way is that much lowerimportance is assigned to state A4 a and thereafter. In this way, thefrequency of image capturing in, for example, time zone 1000 d mayfurther be lowered, or the imaging process per se may be stopped.

In order to embody the timeline as illustrated in FIG. 5 in a real-timeprocess, the key is to determine the importance corresponding to thesuccessive embryo development stages. The determination unit 203according to this embodiment determines the importance of the embryodevelopment stages contained in image data, using analytical result(feature) obtained from the image analysis unit 202. In this way, evenin a long timeline that ranges over several days, an appropriatetimeline for imaging process may be set without overloading the user.Note that the preset of the timeline may be given by a function of thesetting unit 204 in the succeeding stage.

A method of determining the importance by the determination unit 203according to this embodiment will be explained. For example, thedetermination unit 203 determines the importance on the basis of atleast one feature acquired from the image analysis unit 202. Morespecifically, the determination unit 203 may discriminate the embryodevelopment stages by applying the feature acquired from the imageanalysis unit 202, to a learned model that has learned a relationbetween the embryo development stages and the individual features. Thelearned model is a model constructed typically by using the embryodevelopment stages as a response variable, and using the individualfeatures as an explanatory variable. Technique regarding such learningmay be any of known techniques, such as a neural network based on deeplearning or the like.

Alternatively, even in a case where the aforementioned model is notused, the determination unit 203 may determine the importance by usingthe size of the thus acquired feature as an index, and by comparing theindex with a predetermined threshold value. The predetermined thresholdvalue may appropriately be preset so as to be corresponded to theaforementioned embryo development stages. In this way,

Otherwise, the determination unit 203 can be embodied by any of knowntechnologies of image analysis, so long as the importance related toembryo development stages can be determined from image data.

Note that the determination unit 203 may carry out the determinationprocess by using not only a single image data, but also image data inthe frames preceding and succeeding that image data. In this way, theembryo development stages will be determined more exactly, and theaccuracy of determination of importance will be improved.

The determination unit 203 outputs information regarding the thusdetermined importance to the setting unit 204.

(Setting Unit)

The setting unit 204 has a function of providing settings regarding atarget of acquisition of image data in the time-series imaging process,on the basis of the determination result of importance. The settingsregarding a target of acquisition of image data, carried out by thesetting unit 204 according to this embodiment mean two processes, namely(1) setting for target temporal unit, and (2) setting for target ofgeneration (target of acquisition).

(1) Setting for Target Temporal Unit

The setting unit 204 according to this embodiment sets a target temporalunit regarding the target of generation of image data, on the basis of aprocess period for imaging process by the imaging device 10. The settingfor target temporal unit in this context means setting for targettemporal unit by ranking the timeline illustrated in FIG. 5 by theimportance, that is, updating the timeline for imaging process dependingon the importance. The setting unit 204 according to this embodiment canupdate the timeline, corresponding to the thus determined importance.

Now, the setting process of the target temporal unit carried out by thesetting unit 204 will be explained referring to FIG. 6 to FIG. 8. FIG. 6to FIG. 8 are graphs illustrating exemplary updating processes at timepoints of judgment t₁, t₂ and t₃, carried out by the setting unit 204according to this embodiment. Note that in this updating process, theimportance will be explained as being ranked on a three-point scale of“high”, “medium” and “low”. Meanwhile, the determination time pointmeans time point of imaging regarding target image data to be analyzedby the image analysis unit 202. In this example, judgment of state A2and state A3 will be assigned with “high” importance, judgment of stateA will be assigned with “medium” importance, and judgment of state D1and thereafter will be assigned with “low” importance.

First, in timeline TL1 before updating illustrated in FIG. 6, theimportance is uniformly set to “medium” throughout the process period ofimaging process (bar 1001). Now assume that, at determination time pointt₁, the embryo development stage contained in the image data isdetermined to have state A2 (2-cell stage) by the determination unit203, with the importance of “high”. The setting unit 204 then sets atarget temporal unit with the importance of “high”, at the determinationtime point t₁ and thereafter (bar 1002). The setting unit 204 thusyields updated timeline TL2.

Alternatively, the setting unit 204 may further update the timelinehaving been updated once. Assuming that, during the imaging processsustained according to timeline TL2 as illustrated in FIG. 7, the embryodevelopment stage contained in the image data is determined again tohave state A2 a at determination time point t₂ by the determination unit203, with the importance of “high”. Then the setting unit 204 mayfurther extend the target temporal unit with the importance of “high”(bar 1003). In the thus updated timeline TL3, the end point of thetarget temporal unit can further be put off.

Alternatively, assuming that, during the imaging process sustainedaccording to timeline TL3 as illustrated in FIG. 8, the embryodevelopment stage contained in the image data is determined to havestate D1 at determination time point t₃ by the determination unit 203,with the importance of “low”. The setting unit 204 then sets a targettemporal unit with the importance of “low”, at the determination timepoint t₃ and thereafter (bar 1004). In this way, the frequency of imagecapturing and so forth may be reduced or the imaging process per se maybe stopped at determination time point t₃ and thereafter, and thetemporal unit that falls on the time point and thereafter can beexcluded from the target of acquisition of image data.

As described above, by appropriately updating the target temporal unitusing the importance determined by the determination unit 203, enabledis real-time setting for the target of generation in the later-describedimage capturing.

Note that the length of target temporal unit may, for example, be apredetermined length (more specifically, length corresponded tointervals of the time-lapse imaging, for example). Alternatively, thelength of the target temporal unit may appropriately be controlledcorresponding to the embryo development stages (that is, thecell-specific event) determined by the determination unit 203. Stillalternatively, the length of target temporal unit may be controlled onthe basis of the feature calculated in image analysis carried out by theimage analysis unit 202. For an exemplary case where cells afteradministered with a drug need be observed regarding an event such asmedical efficacy, the timing at which the changes will appear over along period of imaging process may vary depending on types of drug.Hence by setting the target temporal unit on the basis of the determinedembryo development stages or calculated feature, the temporal unit withhigh importance (or low importance) may be determined in a moreappropriate manner. Similarly, the length of extended unit may be apredetermined length like the aforementioned length of target temporalunit, or may be set corresponding to the embryo development stagesdetermined by the determination unit 203.

Note that the time points of judgment illustrated in FIG. 6 to FIG. 8are merely for exemplary purposes. For example, the determination unit203 may carry out the determination process assuming all time points ofimaging set on the timeline as the determination time point, or assumingat least one time point selected from a plurality of time points ofimaging as the determination time point. The setting unit 204 canappropriately set the target temporal unit, using the importanceobtained as a result of the determination process carried out by thedetermination unit 203. Also note that, although the target temporalunit corresponded to the importance is not set in timeline TL1 beforeupdating in FIG. 6, the target temporal unit may be set preliminarilydepending on the importance, even for the timeline before beingsubjected to the setting process carried out by the setting unit 204.

(2) Setting for Target of Generation

The setting unit 204 according to this embodiment also sets the targetof generation of image data (target of acquisition) in the time-seriesimaging process. Now the setting for the target of generation of imagedata means setting for the target of generation of image data in thetime-series imaging process, or setting of increase or decrease of datasize of producible image data.

In this embodiment, the setting for target of generation means, forexample, deletion or addition of the timing of imaging in thetime-series imaging process. FIG. 9 is a graph illustrating a firstexample of the setting for the target of generation carried out by thesetting unit 204 according to this embodiment. Referring to FIG. 9, thetimeline regarding the time-series imaging process initially has, asseen in “normal”, the timing of imaging which is set at regularintervals. After the importance is determined by the determination unit203, and the target temporal unit is set according to the importance bythe setting unit 204, the setting unit 204 can update the timing ofimaging depending on the importance.

For example in a timeline named “thin”, the setting unit 204 can makethe setting so as to delete the timing of imaging in the temporal unitof “low” importance. In this way, the image data regarding thecell-specific events which are not important in observation of cells,may be suppressed from being accumulated. In addition, upon switching ofthe temporal unit of “low” importance into the temporal unit of “medium”importance, the setting unit 204 may return the timing of imaging backto the intervals in the initial timeline. This helps to maintain qualityof the acquired image data.

On the other hand in a timeline named “dense”, the setting unit 204 canmake the setting so as to add the timing of imaging in the temporal unitof “high” importance. In this way, the image data regarding events,which are important in observation of cells, may be accumulatedintensively. Alternatively, the setting unit 204 may reduce the timingof imaging from the original plan in the temporal unit of “low”importance, and may increase the timing of imaging from the originalplan in the temporal unit of “high” importance.

Alternatively as explained above, the setting unit 204 may delete thetiming of imaging at a predetermined time point and thereafter, on thebasis of the importance. That is, the setting unit 204 may make settingso as to stop the imaging process at a predetermined time point andthereafter. This helps to suppress unnecessary image data regarding thecell-specific events which need not be observed, from being accumulated.

In addition, in a case where image data of Z-stack image (a plurality ofimages obtainable at one time point by multiple imaging at differentfocal positions) is generable in the time-series imaging process, thesetting for the target of generation may include setting of the numberof shots (number of acquisition) and focal position of the Z-stackimage. FIG. 10 is a graph illustrating a second example of the settingfor the target of generation, carried out by the setting unit 204according to this embodiment. Referring to FIG. 10, the timelineregarding the time-series imaging process initially has, as seen in“normal”, the number of shots of Z-stack image which is set to X (>1)shots. After the importance is determined by the determination unit 203,and the target temporal unit is set according to the importance by thesetting unit 204, the setting unit 204 can update the number of shots ofZ-stack image depending on the importance.

For example, in the “thin” timeline, the setting unit 204 can make thesetting so as to decrease the number of shots of Z-stack image in thetemporal unit of “low” importance (for example, 1 shot). At the sametime, also the focal position of producible Z-stack image can be setdepending on the preset number of shots. Specific setting of focalposition can be made depending on types of cell assumed as the target tobe observed, cell-specific event and so forth. In this way, the imagedata regarding cell-specific events which are not important inobservation of cells, may be suppressed from being accumulated.

Meanwhile, in the “dense” timeline, the setting unit 204 can make thesetting so as to increase the number of shots of Z-stack image in thetemporal unit of “high” importance. At the same time, also the focalposition of producible Z-stack image can be set depending on the presetnumber of shots. In this way, the image data regarding events, which areimportant in observation of cells, may be accumulated intensively.Alternatively, the setting unit 204 may reduce the number of shots fromthe original plan in the temporal unit of “low” importance, and mayincrease the number of shots from the original plan in the temporal unitof “high” importance.

In addition in this embodiment, also a process regarding increase ordecrease of data size of the image data to be produced can take place.Such increase or decrease of data size of image data to be producedmeans increase or decrease of resolution or compressibility of thegenerable image data. More specifically, the setting unit 204 may sethigh resolution or low compressibility for image data generable in thetemporal unit with high importance. Meanwhile, the setting unit 204 mayset low resolution and high compressibility for image data generable inthe temporal unit with low importance.

FIG. 11 is a graph illustrating a third example of the setting for thetarget of generation carried out by the setting unit 204 according tothis embodiment. Referring to FIG. 11, the timeline regarding thetime-series imaging process initially has, as seen in “normal”, theresolution of the generable image data which is set to mediumresolution.

Then after the importance is determined by the determination unit 203,and the target temporal unit is set according to the importance by thesetting unit 204, the setting unit 204 can update the resolution of thegenerable image data depending on the importance. More specifically, asseen in an “updated” timeline illustrated in FIG. 11, the setting unit204 can set the resolution to low resolution (or medium resolution) inthe temporal unit with “low” importance. Meanwhile, the setting unit 204can set the resolution to medium resolution (or high resolution) in thetemporal unit with “high” importance. In this way, the data size of theimage data regarding the cell-specific events, which are not importantin observation of cells, may be reduced, and the quality of image dataregarding events, which are important in observation of cells, may beenhanced preferentially.

As described above, as a result of the (updating) setting of thetimeline corresponded to the importance carried out by the setting unit204, the image data primarily containing the time-lapse image thatrepresents a target cell-specific event to be observed may be acquiredin an exact and efficient manner.

Note that, there may be a case where the setting process of timeline bythe setting unit 204 will not be carried out, depending on thedetermination result given by the determination unit 203. For example,in a case where the determination result on the importance given by thedetermination unit 203 is identical to the immediately previousdetermination result, and the target temporal unit and so forth need notbe updated, the setting process of timeline by the setting unit 204 maybe omissible.

The setting unit 204 outputs information that is based on the settingthus acquired in the setting process, to the output control unit 205.

(Output Control Unit)

The output control unit 205 has a function of controlling an output as aresult of processes in the control unit 200. For example, the outputcontrol unit 205 according to this embodiment controls to outputinformation that is based on the setting acquired from the setting unit204. More specifically, the output control unit 205 outputs theinformation based on the aforementioned setting to the imaging controlunit 102 of the imaging device 10, through the communication unit 210.Such information based on the setting include information regarding thetimeline of the time-series imaging process updated by the setting unit204. Upon acquisition of the information, the imaging control unit 102carries out the imaging process according to the updated timeline. Thatis, the imaging control unit 102 can carry out the imaging process,according to the timing of imaging, the number of shots of Z-stackimage, or image quality (increase or decrease of data size) which arespecified by the timeline.

Alternatively, the output control unit 205 may output the informationbased on the setting to the image acquisition unit 201. In this case,the image acquisition unit 201 will acquire, from the imaging device 10,only the image data preset as a target of acquisition, according to theupdated timeline. This consequently makes it possible to reduce thetarget image data assumed as a target of generation, or to reduce thedata size of the produced image data, although the imaging process perse is not controlled.

The construction of the control unit 200 according to this embodimenthas been explained. As the individual functional units equipped to suchcontrol unit 200 repetitively carry out the determination process forimportance using image data, and the setting process for target ofgeneration, concurrently with the imaging process carried out by theimaging device 10, the timeline corresponded to the importance issequentially updated to control the imaging process. This consequentlymakes it possible to store the image data that represents acell-specific event which is desired to be observed, in an efficient andexact manner, without overloading the user even in prolongedobservation. In addition, the target of generation of image data mayoptionally be decreased or increased, making it possible to suppressphototoxicity possibly exerted on the cell during image capturing of thecell.

<2.2. Exemplary Process>

The configuration and the functions of the information processing device20 according to the embodiment have been described above. Next, anexemplary process performed by the information processing device 20according to the embodiment will be described referring to FIG. 12.

FIG. 12 is a flowchart illustrating an exemplary process performed bythe information processing device 20 according to the first embodimentof the present disclosure. The flowchart in FIG. 12 illustrates anexemplary flow of processes initiated at the start point of the imagingprocess carried out by the imaging device 10, followed by acquisition ofimage data from the imaging device 10 by the information processingdevice 20, determination of the importance on the basis of analyticalresult (feature) of image data, and setting for the target of generationusing the determination result of importance.

First, prior to the imaging process, the setting unit 204 performsinitial setting of timeline in the imaging process (step S101). Thetimeline to be initially set is a timeline typically shown in FIG. 6 astimeline TL1, whose target temporal unit has not yet been updated on thebasis of the importance. Next, the output control unit 205 outputs thetimeline to the imaging control unit 102 of the imaging device 10 (stepS103). The imaging control unit 102 controls the imaging process on thebasis of the acquired timeline.

Next, the image acquisition unit 201 acquires, from the imaging device10, the image data regarding cells produced by the imaging process (stepS105). Next, the image analysis unit 202 analyzes the acquired imagedata, and calculate a feature as an analytical result (step S107).

Next, the determination unit 203 discriminates a cell-specific eventexpressed in the image data using the calculated feature, and determinesthe importance (step S109). The setting unit 204 determines whether thetimeline is updated or not, on the basis of the determination result ofimportance (step S111). If the timeline need be updated (S111/YES), thesetting unit 204 updates the timeline (step S113).

If the imaging process is sustained (step S115/YES), output control unit205 outputs the updated timeline to the imaging device 10 (S103). Whilethe imaging process is carried out by the imaging device 10, theinformation processing device 20 appropriately repeats the processesshown in step S103 to step S115. This enables updating of the timelinein a real-time manner.

The first embodiment of the present disclosure has been explained.

3. SECOND EMBODIMENT

Next, a second embodiment of the present disclosure will be explainedreferring to FIG. 13 and FIG. 14. In the information processing system 1according to this embodiment, carried out is a process regarding settingof a target of storage of image data produced in the imaging processcarried out by the imaging device 10. That is, in the informationprocessing system 1 according to the first embodiment of the presentdisclosure, the image data which will be generable in the future hasbeen processed concurrently with the imaging process carried out by theimaging device 10. In contrast in the information processing system 1according to this embodiment, the image data that is produced and onceaccumulated in the storage unit 220 or the like can directly beprocessed. This enables selective storage of the image data which isconsidered to be important, among from the image data captured in theimaging process. The image data may therefore be stored efficiently andmore exactly, without overloading the user.

Note that the process held by the information processing device 20according to this embodiment may take place after the imaging device 10completed the imaging process, or may take place concurrently with theimaging process by the imaging device 10, targeting the image datahaving been produced in advance and stored in the storage unit 220 orthe like.

<3.1. Exemplary Configuration>

Configuration of the information processing device 20 according to thisembodiment is identical to the configuration of the informationprocessing device 20 according to the first embodiment of the presentdisclosure illustrated in FIG. 2. The paragraphs below will explainfunctions that are different from those of the information processingdevice 20 according to the first embodiment of the present disclosure.

(Image Acquisition Unit)

The image acquisition unit 201 according to this embodiment has afunction of acquiring image data produced in the time-series imagingprocess targeting a cell. Such image data is stored in the storage unit220. The image acquisition unit 201 may sequentially acquire the imagedata from the imaging device 10 when the imaging process by the imagingdevice 10 is underway, or may acquire the image data from the imagingdevice 10 after completion of the imaging process by the imaging device10. The thus acquired image data is subjected to image analysis by theimage analysis unit 202, from which a feature regarding the image datacan be calculated.

(Determination Unit)

The determination unit 203 can determine the importance on the basis ofthe thus calculated feature. Technique for the determination process bythe determination unit 203 is identical to that in the first embodimentof the present disclosure.

The determination process by the determination unit 203 according tothis embodiment is carried out for each of the plurality of image dataproduced by the time-series imaging process. Hence the setting unit 204in the succeeding stage can rank the timeline of the time-series imagingprocess (the timeline herein means time-series information of producedimage data after completion of the time-series imaging process, such astime point of generation, or the number of shots of image data)according to the importance, and can preset the target temporal unit.

Note that the image data determined by the determination unit 203 may beall image data produced in the time-series imaging process, or may beimage data partially extracted from the timeline of the imaging process.

Alternatively, the determination unit 203 may correlate the image datahaving been subjected to the determination process, with thedetermination result of importance. Hence in the setting process oftarget temporal unit in the setting unit 204 in the succeeding stage, itnow becomes possible to preset the target temporal unit on the basis ofthe time point of generation of image data.

(Setting Unit)

The setting unit 204 has a function of performing setting for the targetof acquisition of image data in the time-series imaging process, on thebasis of determination result of importance. The setting for the targetof acquisition of image data performed by the setting unit 204 accordingto this embodiment means two types of process, namely (1) setting fortarget temporal unit, and (2) setting for target of storage (target ofacquisition).

(1) Setting for Target Temporal Unit

The setting unit 204 according to this embodiment sets the targettemporal unit regarding setting for the target of storage of theproduced image data, on the basis of the process period of imagingprocess carried out by the imaging device 10. Meaning of the setting forthe target temporal unit is same as that in the first embodiment of thepresent disclosure.

The setting unit 204 may sequentially preset the target temporal unit,on the basis of the importance determined using the sequentiallyacquired image data, in the same way as in the first embodiment of thepresent disclosure. Such setting process can be carried out in a casewhere the imaging process and the storage process of image data takeplace concurrently. Alternatively, the setting unit 204 according tothis embodiment may preset the target temporal unit, on the basis of thetime point of generation of image data and the importance correlated tothe image data, as described above. In this way, the target temporalunit ranked by the importance can be preset more finely.

(2) Setting for Target of Storage

Also, the setting unit 204 according to this embodiment presets thetarget of storage of image data in the time-series imaging process. Nowthe setting for target of storage of image data means the setting for atarget image data (target of storage) to be stored, among from the imagedata produced in the time-series imaging process, or setting fordecreasing the data size of the produced image data.

The setting for target of storage in this embodiment means, for example,reduction of the target of storage, among from the image data producedin the imaging process according to the timeline. FIG. 13 is a graphillustrating an exemplary setting for the target of generation carriedout by the setting unit 204 according to this embodiment. Referring toFIG. 13, the image data is produced at regular intervals of timing ofimaging in the time-series imaging process. Upon completion of theimaging process, the importance is determined by the determination unit203, and the target temporal unit is preset by the setting unit 204.Then in the example illustrated in FIG. 13, the setting unit 204 presetsa part of image data in a temporal unit with “low” importance, as atarget of reduction. Then the image data preset as the target ofreduction can be deleted from the storage unit 220, by the outputcontrol unit 205 in the succeeding stage. In this way, the image dataregarding the cell-specific events which are not important inobservation of cells, may be suppressed from being accumulated.

In addition, similarly to the first embodiment of the presentdisclosure, in a case where image data of Z-stack image is generable inthe time-series imaging process, the setting for the target of storagemay include the number of shots (number of acquisition) and focalposition of the Z-stack image. For example, the setting unit 204 canmake the setting so as to reduce the number of storage (for example, 1shot) of image data in the target temporal unit with “low” importance.In this process, the setting can be made also regarding that the imagedata captured at which focal position will be stored. Specific settingfor focal position can be made typically depending on types orcell-specific events of cell assumed as the target to be observed. Inthis way, the image data regarding the cell-specific events which arenot important in observation of cells, may be suppressed from beingaccumulated.

In addition, also a process regarding reduction of data size of thestored image data can be carried out, in the same way as in the firstembodiment of the present disclosure. The reduction of data size of thestored image data means, for example, reduction of resolution of thestored image data, or increase of the compressibility. Morespecifically, the setting unit 204 may set low resolution or highcompressibility for image data stored in the temporal unit with lowimportance. In this way, the data size of image data regarding thecell-specific events which are not important in observation of cells,may be reduced.

As described above, as a result of the setting for the target temporalunit corresponded to the importance carried out by the setting unit 204,the image data primarily containing the time-lapse image that representsthe target cell-specific event to be observed may be acquired in anexact and efficient manner.

(Output Control Unit)

The output control unit 205 according to this embodiment controls anoutput process on the basis of preset items regarding the target ofstorage set by the setting unit 204. For example, the output controlunit 205 may make control so as to delete image data assumed as thetarget of deletion, among from the image data accumulated in the storageunit 220. Alternatively, the output control unit 205 may make controlfor reducing the data size of image data assumed as the target ofstorage.

Note that the output control unit 205 may output information regardingsetting for the target of storage to the imaging device 10. In this way,the imaging control unit 102 can delete image data assumed as the targetof deletion, among from the image data accumulated in the unillustratedstorage unit of the imaging device 10.

The configuration of the control unit 200 according to this embodimenthas been explained. With the aid of the individual functional unitsequipped to such control unit 200, the setting process for the target ofstorage of image data, accumulated in the imaging process carried out bythe imaging device 10, takes place on the basis of the importanceobtained as a result of the determination process using the image data.In this way, it becomes possible to efficiently and exactly store onlyimage data that represents the cell-specific events that are desired tobe observed, among from a huge volume of image data having beenaccumulated over a long period, without overloading the user.

<3.2. Exemplary Processing>

The configuration and the functions of the information processing device20 according to the embodiment have been described above. Next, anexemplary process performed by the information processing device 20according to the embodiment will be described referring to FIG. 14.

FIG. 14 is a flowchart illustrating an exemplary process performed by aninformation processing device 20 according to the second embodiment ofthe present disclosure. The flowchart illustrated in FIG. 14 illustratesan exemplary flow of processes in which, upon completion of the imagingprocess by the imaging device 10, the information processing device 20acquires a series of image data in the time-series imaging process fromthe imaging device 10, determines the importance according to theanalytical result (feature) of the individual image data, and presetsthe target of storage using the determination result of importance.

First, upon completion of the imaging process, the image acquisitionunit 201 acquires the image data regarding cells, having been producedby the imaging process and then accumulated, from the storage unit 220(or from the imaging device 10) (step S201). Next, the image analysisunit 202 analyzes the acquired image data, and calculate a feature as ananalytical result (step S203).

Next, the determination unit 203 discriminates a cell-specific eventexpressed in each of the image data using the calculated feature, anddetermines the importance (step S205). Next, the setting unit 204carries out a process regarding setting for the target of storage, onthe basis of the determination result of importance (step S207). Theoutput control unit 205 then carries out a process for image data, suchas deletion of image data assumed as the target of deletion, orreduction of data size of image data assumed as the target of storage,depending on the items of setting regarding the target of storage (stepS209).

An exemplary process carried out by the information processing device 20according to this embodiment has been explained. Note that the flowchartillustrated in FIG. 14 is on the premise of a post-process that takesplace after all image data were acquired. For example, by appropriatelyrepeating the processes assigned to step S201 to S209, these processesmay be carried out concurrently with the imaging process, in a real-timemanner.

The second embodiment of the present disclosure has been explainedabove.

4. EXEMPLARY HARDWARE CONFIGURATION

Next, with reference to FIG. 15, a hardware configuration of aninformation processing device according to an embodiment of the presentdisclosure is described. FIG. 15 is a block diagram illustrating anexemplary hardware configuration of the information processing deviceaccording to the embodiment of the present disclosure. An illustratedinformation processing device 900 can realize the information processingdevice 20 in the above described embodiment.

The information processing device 900 includes a CPU 901, read onlymemory (ROM) 903, and random access memory (RAM) 905. In addition, theinformation processing device 900 may include a host bus 907, a bridge909, an external bus 911, an interface 913, an input device 915, anoutput device 917, a storage device 919, a drive 921, a connection port925, and a communication device 929. The information processing device900 may include a processing circuit such as a digital signal processor(DSP) or an application-specific integrated circuit (ASIC), instead ofor in addition to the CPU 901.

The CPU 901 functions as an arithmetic processing device and a controldevice, and controls the overall operation or a part of the operation ofthe information processing device 900 according to various programsrecorded in the ROM 903, the RAM 905, the storage device 919, or aremovable recording medium 923. For example, the CPU 901 controlsoverall operations of respective function units included in theinformation processing device 20 of the above-described embodiment. TheROM 903 stores programs, operation parameters, and the like used by theCPU 901. The RAM 905 transiently stores programs used when the CPU 901is executed, and parameters that change as appropriate when executingsuch programs. The CPU 901, the ROM 903, and the RAM 905 are connectedwith each other via the host bus 907 configured from an internal bussuch as a CPU bus or the like. The host bus 907 is connected to theexternal bus 911 such as a Peripheral Component Interconnect/Interface(PCI) bus via the bridge 909.

The input device 915 is a device operated by a user such as a mouse, akeyboard, a touchscreen, a button, a switch, and a lever. The inputdevice 915 may be a remote control device that uses, for example,infrared radiation and another type of radio waves. Alternatively, theinput device 915 may be an external connection device 927 such as amobile phone that corresponds to an operation of the informationprocessing device 900. The input device 915 includes an input controlcircuit that generates input signals on the basis of information whichis input by a user to output the generated input signals to the CPU 901.The user inputs various types of data and indicates a processingoperation to the information processing device 900 by operating theinput device 915.

The output device 917 includes a device that can visually or audiblyreport acquired information to a user. The output device 917 may be, forexample, a display device such as an LCD, a PDP, and an OELD, an audiooutput device such as a speaker and a headphone, and a printer. Theoutput device 917 outputs a result obtained through a process performedby the information processing device 900, in the form of text or videosuch as an image, or sounds such as audio sounds.

The storage device 919 is a device for data storage that is an exemplarystorage unit of the information processing device 900. The storagedevice 919 includes, for example, a magnetic storage device such as ahard disk drive (HDD), a semiconductor storage device, an opticalstorage device, or a magneto-optical storage device. The storage device919 stores therein the programs and various data executed by the CPU901, and various data acquired from an outside.

The drive 921 is a reader/writer for the removable recording medium 923such as a magnetic disk, an optical disc, a magneto-optical disk, and asemiconductor memory, and built in or externally attached to theinformation processing device 900. The drive 921 reads out informationrecorded on the mounted removable recording medium 923, and outputs theinformation to the RAM 905. The drive 921 writes the record into themounted removable recording medium 923.

The connection port 925 is a port used to directly connect devices tothe information processing device 900. The connection port 925 may be aUniversal Serial Bus (USB) port, an IEEE1394 port, or a Small ComputerSystem Interface (SCSI) port, for example. The connection port 925 mayalso be an RS-232C port, an optical audio terminal, a High-DefinitionMultimedia Interface (HDMI (registered trademark)) port, and so on. Theconnection of the external connection device 927 to the connection port925 makes it possible to exchange various kinds of data between theinformation processing device 900 and the external connection device927.

The communication device 929 is a communication interface including, forexample, a communication device for connection to a communicationnetwork NW. The communication device 929 may be, for example, a wired orwireless local area network (LAN), Bluetooth (registered trademark), ora communication card for a wireless USB (WUSB). The communication device929 may also be, for example, a router for optical communication, arouter for asymmetric digital subscriber line (ADSL), or a modem forvarious types of communication. For example, the communication device929 transmits and receives signals in the Internet or transits signalsto and receives signals from another communication device by using apredetermined protocol such as TCP/IP. The communication network NW towhich the communication device 929 connects is a network establishedthrough wired or wireless connection. The communication network NW is,for example, the Internet, a home LAN, infrared communication, radiowave communication, or satellite communication.

Note that the CPU 901, the ROM 903 and the RAM 905 and so forth canenable the functions of the control unit 200 according to theembodiment. Meanwhile the storage device 919 can enable the function ofthe storage unit 220 according to the embodiment. Moreover, at leasteither the connection port 925 or the communication device 929 canenable the function of the communication unit 210 according to theembodiment.

The example of the hardware configuration of the information processingdevice 900 has been introduced.

5. CONCLUSION

The preferred embodiment (s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

For example, although the information processing system 1 is configuredto be provided with the imaging device 10 and information processingdevice 20 in the above-described embodiment, the present technology isnot limited thereto. For example, the imaging device 10 may have thefunction of the information processing device 20 (For example, functionsregarding image analysis process, determination process, and settingprocess or the like). In this case, the information processing system 1is embodied by the imaging device 10. In addition, the informationprocessing device 20 may have the function of the imaging device 10(imaging function). In this case, the information processing system 1 isembodied by the information processing device 20. Further, the imagingdevice 10 may have a part of the function of the information processingdevice 20, and the information processing device 20 may have a part ofthe function of the imaging device 10.

Note that the information processing system 1 according to theindividual embodiments assumed embryo, which is an exemplary cell, as atarget of application. The embryo as a target of application is notlimited to human embryo, but may also be embryo of mammals such asmouse, embryo of non-mammalian animal, or embryo of non-animalmulticellular organism. Also note, as described above, the informationprocessing system 1 according to one embodiment of the presentdisclosure is applicable not only to embryo, but also to a wide varietyof cells including cancer cell and iPS cell.

The steps in the processes performed by the information processingdevice in the present specification may not necessarily be processedchronologically in the orders described in the flowcharts. For example,the steps in the processes performed by the information processingdevice may be processed in different orders from the orders described inthe flowcharts or may be processed in parallel.

Also, a computer program causing hardware such as the CPU, the ROM, andthe RAM included in the information processing device to carry out theequivalent functions as the above-described configuration of theinformation processing device can be generated. Also, a readablerecording medium having the computer program stored therein can beprovided.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing device including:

a determination unit that determines importance related to acell-specific event of a cell, using image data obtained from atime-series imaging process targeting the cell; and

a control unit that controls a process regarding setting for a target ofacquisition of image data in the time-series imaging process, on thebasis of a determination result of the importance.

(2)

The information processing device according to (1),

in which the process regarding the setting for a target of acquisitionof image data includes a process that presets the target of acquisitionof image data in the time-series imaging process, on the basis of thedetermination result of importance.

(3)

The information processing device according to (2),

in which the process that presets the target of acquisition of imagedata includes a process that presets time point of generation, in thetime-series imaging process, of the image data assumed as the target ofacquisition.

(4)

The information processing device according to (2) or (3),

in which, in a case where a plurality of image data with different focalpositions is generable at one time point in the time-series imagingprocess,

the process that presets the target of acquisition of image dataincludes a process that presets the number of acquisition of image dataat the one time point and/or the focal positions.

(5)

The information processing device according to any one of (1) to (4),

in which the process regarding the setting for a target of acquisitionof image data includes a process that deletes the target of acquisitionin the time-series imaging process.

(6)

The information processing device according to any one of (1) to (5),

in which the process regarding the setting for a target of acquisitionof image data includes a process that excludes image data acquired at apredetermined time point and thereafter in the time-series imagingprocess, from the target of acquisition.

(7)

The information processing device according to any one of (1) to (6),

in which the process regarding the setting for a target of acquisitionof image data controls a process regarding increase or decrease of datasize of the image data preset as a target of acquisition, on the basisof the determination result of importance.

(8)

The information processing device according to any one of (1) to (7),

in which the process regarding the setting for a target of acquisitionof image data includes a process that updates time-series informationthat specifies the target of acquisition of image data in thetime-series imaging process.

(9)

The information processing device according to (8),

in which the process that updates time-series information includes aprocess that deletes the target of acquisition of image data from thetime-series information, and/or a process that adds the target ofacquisition of image data to the time-series information.

(10)

The information processing device according to any one of (1) to (9),

in which the process regarding the setting for a target of acquisitionof image data includes a process regarding the setting for a target ofstorage of image data generated by the time-series imaging process.

(11)

The information processing device according to any one of (1) to (10),

in which the control unit presets a target temporal unit targeted by theprocess regarding the setting for a target of acquisition of image data,out from a process period of the time-series imaging process,corresponding to the importance.

(12)

The information processing device according to (11),

in which length of the target temporal unit includes length determinedcorresponding to the cell-specific event related to the determinedimportance.

(13)

The information processing device according to (11) or (12),

in which length of the target temporal unit includes length determinedon the basis of feature obtained by analyzing the image data.

(14)

The information processing device according to any one of (1) to (13),

in which the determination unit determines the importance using afeature obtained by analyzing the image data.

(15)

The information processing device according to (14),

in which the determination unit discriminates a cell-specific eventregarding the cell contained in the image data, on the basis of thefeature, and determines the importance on the basis of thediscrimination result.

(16)

The information processing device according to (15),

in which the importance is estimated using a learned model that haslearned a preliminarily acquired relation between the cell-specificevent retarding the cell contained in the image data, and featureobtained by analyzing the image data.

(17)

The information processing device according to any one of (13) to (16),

in which the feature includes a feature estimated on the basis of atleast any one of morphology of image and kinetics of the cell containedin the image data, and pixel of the image data.

(18)

An information processing method including by a processor:

determining importance related to a cell-specific event of a cell, usingimage data obtained from a time-series imaging process targeting thecell; and

controlling a process regarding setting for a target of acquisition ofimage data in the time-series imaging process, on the basis of adetermination result of the importance.

(19)

An information processing system including:

an imaging device that includes

-   -   an imaging unit that produces an image by image capturing;        and

an information processing device that includes

-   -   a determination unit that determines importance related to a        cell-specific event of a cell, using image data obtained from a        time-series imaging process targeting the cell by the imaging        unit; and    -   a control unit that controls a process regarding setting for a        target of acquisition of image data in the time-series imaging        process, on the basis of a determination result of the        importance.

REFERENCE SIGNS LIST

-   1 information processing system-   10 imaging device-   20 information processing device-   101 imaging unit-   102 imaging control unit-   200 control unit-   201 image acquisition unit-   202 image analysis unit-   203 determination unit-   204 setting unit-   205 output control unit-   210 communication unit-   220 storage unit

The invention claimed is:
 1. An information processing devicecomprising: circuitry configured to: determine importance of acell-specific event of a cell, based on image data obtained from atime-series imaging process that targets the cell, wherein adetermination result of the determined importance is one of a firstimportance and a second importance; control a setting process for atarget of acquisition of image data in the time-series imaging process,based on the determination result of the determined importance; rank atimeline of a process period of the time-series imaging process based onthe determination result of the determined importance; and preset a timezone, of the ranked timeline, that has the first importance as a targettemporal unit, wherein the setting process for the target of acquisitionof image data targets the target temporal unit.
 2. The informationprocessing device according to claim 1, wherein the first importance isone of a high importance and a low importance.
 3. The informationprocessing device according to claim 1, wherein the setting process forthe target of acquisition of image data includes a setting process topreset time point of generation, in the time-series imaging process, ofthe image data assumed as the target of acquisition.
 4. The informationprocessing device according to claim 1, wherein the setting process forthe target of acquisition of image data includes a setting process topreset a number of acquisitions of image data with different focalpositions at one time point in the time-series imaging process, andwherein a plurality of image data with the different focal positions isgenerated at the one time point in the time-series imaging process basedon the preset number of acquisitions of image data.
 5. The informationprocessing device according to claim 1, wherein the setting process forthe target of acquisition of image data includes a setting process todelete the target of acquisition in the time-series imaging process. 6.The information processing device according to claim 1, wherein thesetting process for the target of acquisition of image data includes asetting process to exclude image data acquired at a predetermined timepoint and thereafter in the time-series imaging process, from the targetof acquisition.
 7. The information processing device according to claim1, wherein the setting process for the target of acquisition of imagedata includes a setting process to increase or decrease data size of theimage data that is preset as the target of acquisition in the targettemporal unit based on the determination result of the determinedimportance.
 8. The information processing device according to claim 1,wherein the setting process for the target of acquisition of image dataincludes a setting process to update time-series information thatspecifies the target of acquisition of image data in the time-seriesimaging process.
 9. The information processing device according to claim8, wherein the setting process to update the time-series informationincludes a first setting process to delete the target of acquisition ofimage data from the time-series information, and/or a second settingprocess to add the target of acquisition of image data to thetime-series information.
 10. The information processing device accordingto claim 1, wherein the setting process for the target of acquisition ofimage data includes a setting process for a target of storage of imagedata generated by the time-series imaging process.
 11. The informationprocessing device according to claim 1, wherein the circuitry is furtherconfigured to: update the target temporal unit based on thedetermination result of the determined importance, wherein the update ofthe target temporal unit corresponds to a control of a length of thetarget temporal unit; and update the timeline based on the update of thetarget temporal unit.
 12. The information processing device according toclaim 1, wherein a length of the target temporal unit includes a lengthof the cell-specific event related to the determination result of thedetermined importance.
 13. The information processing device accordingto claim 1, wherein a length of the target temporal unit includes alength determined based on a feature obtained by analysis of the imagedata.
 14. The information processing device according to claim 1,wherein the circuitry is further configured to determine the importancebased on a feature obtained by analysis of the image data.
 15. Theinformation processing device according to claim 14, wherein thecircuitry is further configured to discriminate the cell-specific eventof the cell contained in the image data, based on the feature, andwherein the determination of the importance is based on a discriminationresult.
 16. The information processing device according to claim 15,wherein the importance is determined based on a learned model that haslearned a preliminarily acquired relation between the cell-specificevent of the cell contained in the image data, and the feature obtainedby analysis of the image data.
 17. The information processing deviceaccording to claim 13, wherein the feature includes a feature obtainedbased on at least any one of morphology of image and kinetics of thecell contained in the image data, and pixel of the image data.
 18. Aninformation processing method comprising: determining, by circuitry,importance of a cell-specific event of a cell, based on image dataobtained from a time-series imaging process that targets the cell,wherein a determination result of the determined importance is one of afirst importance and a second importance; controlling, by the circuitry,a setting process for a target of acquisition of image data in thetime-series imaging process, based on the determination result of thedetermined importance; ranking, by the circuitry, a timeline of aprocess period of the time-series imaging process based on thedetermination result of the determined importance; and presetting, bythe circuitry, a time zone of the ranked timeline that has the firstimportance as a target temporal unit, wherein the setting process forthe target of acquisition of image data targets the target temporalunit.
 19. An information processing system comprising: an imaging devicethat includes first circuitry configured to produce an image by imagecapturing; and an information processing device that includes secondcircuitry configured to: determine importance of a cell-specific eventof a cell, based on image data obtained from a time-series imagingprocess that targets the cell, wherein a determination result of thedetermined importance is one of a first importance and a secondimportance; control a setting process for a target of acquisition ofimage data in the time-series imaging process, based on thedetermination result of the determined importance; rank a timeline of aprocess period of the time-series imaging process based on thedetermination result of the determined importance; and preset a timezone, of the ranked timeline, that has the first importance as a targettemporal unit, wherein the setting process for the target of acquisitionof image data targets the target temporal unit.